Why

AddSchemaUpdate.last-column-id is deprecated and optional in the REST OpenAPI spec. Java already ignores this field when parsing add-schema updates and computes the table-level last column ID internally.

However, Java still serializes the deprecated field using schema.highestFieldId(). After dropping the column with the highest field ID, this value can decrease even though table metadata last-column-id must not decrease. REST catalog implementations that validate the deprecated request field can reject otherwise valid schema evolution updates, as reported in #13850.

This does not remove table metadata last-column-id; it only stops sending the deprecated optional field in AddSchemaUpdate. Field-id conflict validation is still carried by AssertLastAssignedFieldId requirements.

Compatibility notes

• The REST OpenAPI spec marks AddSchemaUpdate.last-column-id as deprecated and optional, and says that when it is omitted it will be computed on the server side: https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L3116-L3135
• Apache Iceberg's REST server path validates update requirements before applying metadata updates, and TableMetadata.Builder.addSchema preserves the table-level last column ID with Math.max(lastColumnId, schema.highestFieldId()): https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java#L613-L659 and https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1136-L1138
• Nessie already models this field as optional (OptionalInt) and its changelog notes the AddSchema lastColumnId deprecation/optional change. When the value is omitted, Nessie computes from the schema max field ID and only increases the stored table-level value: https://github.com/projectnessie/nessie/blob/8f3a35c103ccf98fad9a95c22f2cf50993026a7f/catalog/format/iceberg/src/main/java/org/projectnessie/catalog/formats/iceberg/rest/IcebergMetadataUpdate.java#L348-L371 and https://github.com/projectnessie/nessie/blob/8f3a35c103ccf98fad9a95c22f2cf50993026a7f/catalog/format/iceberg/src/main/java/org/projectnessie/catalog/formats/iceberg/nessie/NessieModelIceberg.java#L1198-L1250
• Polaris uses Iceberg's UpdateTableRequest/MetadataUpdate model for commits and applies updates via MetadataUpdate.applyTo(...), so it follows the Iceberg parser/model behavior instead of requiring the deprecated request field. Its copied REST spec also marks the field deprecated and optional: https://github.com/apache/polaris/blob/e3f26cf69f6ee189328f1b7adb90ec83cbbb26d9/api/iceberg-service/src/main/java/org/apache/polaris/service/types/CommitTableRequest.java#L19-L23 and https://github.com/apache/polaris/blob/e3f26cf69f6ee189328f1b7adb90ec83cbbb26d9/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/CatalogHandlerUtils.java#L408-L522 and https://github.com/apache/polaris/blob/e3f26cf69f6ee189328f1b7adb90ec83cbbb26d9/spec/iceberg-rest-catalog-open-api.yaml#L2969-L2988
• The tabulario/iceberg-rest reference image is a thin wrapper around Apache Iceberg's RESTCatalogAdapter, so it does not add a separate last-column-id validator. Its exact behavior is tied to the bundled Iceberg version, which is why the fix belongs in Iceberg's REST metadata update serialization/model: https://github.com/databricks/iceberg-rest-image/blob/62a5078032e0675d34b377d2ead2eea814c1da48/src/main/java/org/apache/iceberg/rest/RESTServerCatalogAdapter.java#L28-L43

Catalogs that still send the deprecated field remain supported because parsing is unchanged. The omission path is part of the REST contract; server implementations that need the table-level last assigned field ID should use existing table metadata and/or AssertLastAssignedFieldId requirements rather than depending on the deprecated AddSchemaUpdate field.

What changed

• Stop serializing deprecated last-column-id for add-schema metadata updates.
• Keep parsing legacy payloads that still include last-column-id.
• Add request serialization coverage for an add-schema update representing the schema after dropping the highest field ID.

Testing

• JAVA_HOME=$(/usr/libexec/java_home -v 21) ./gradlew :iceberg-core:test --tests org.apache.iceberg.TestMetadataUpdateParser --tests org.apache.iceberg.rest.requests.TestUpdateTableRequestParser --tests org.apache.iceberg.rest.TestCatalogHandlers --console=plain --warning-mode=none
• JAVA_HOME=$(/usr/libexec/java_home -v 21) ./gradlew :iceberg-core:spotlessJavaCheck --console=plain --warning-mode=none